Animated Interface Agent Applying Atms-Based Multimodal Input Interpretation

Two requirements should be met in order to develop a practical multimodal interface system, i.e., (1) integration of delayed arrival of data, and (2) elimination of ambiguity in recognition results of each modality. This paper presents an e cient and generic methodology for interpretation of multimodal input to satisfy these requirements. The proposed methodology can integrate delayed-arrival data satisfactorily and e ciently interpret multimodal input that contains ambiguity. In the input interpretation, the multimodal interpretation process is regarded as hypothetical reasoning and the control mechanism of interpretation is formalized by applying ATMS (Assumption-based Truth Maintenance System). The proposed method is applied to an interface agent system that accepts multimodal input consisting of voice and direct indication gesture on a touch display. The system communicates to the user through a human like interface agent's 3D motion image with facial expressions, gestures and a synthesized voice.